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ED Proposed Regulations to Title II of HEA 

Comments from the Data Quality Campaign 


Secretary Duncan, 


The Data Quality Campaign (DQC) appreciates the opportunity to submit comments on the US 
Department of Education's (ED) proposed regulations to implement requirements for the teacher 
preparation program accountability system under Title II of the Higher Education Act of 1965 (HEA), 
Docket ID ED-2014-OPE-0057. These proposed regulations create both an opportunity to change the 
role of data in teacher preparation and a means for states to act on information that they have, but 
have yet to use in a meaningful way. ED should encourage states to focus on creating plans that are 
focused on using data as a tool for continuous improvement and transparency, not only for 
accountability and compliance. 


In Summary: 

States are uniquely positioned to provide feedback to teacher preparation programs (TPPs) on the 
effectiveness of the teachers they train, and to provide information about the quality of TPPs to the 
public. States have been building the significant data capacity to reliably and securely link teachers with 
their students' achievement and growth data with states' TPPs. States will need to demonstrate new 
leadership in being the source for transparency on TPP quality, as well as in creating a culture of using 
data for continuous improvement. 

As such, DQC responds to ED's proposed regulations as follows: 

• Trust in the validity and reliability of the data used to measure TPP quality is critical, and 
therefore states must have in place a high-quality teacher-student data link (TSDL). 

• Many states currently implement a high-quality TSDL, and all states have implemented at least a 
basic TSDL needed to provide feedback to TPPs and data to the public. 

• TPPs should have information about their graduates for the purposes of continuously improving 
educator training. 

• Data about TPP quality should be reported at the program level rather than only at the 
institution level. 

• Multiple stakeholders should be engaged in the process of developing the system for sharing 
and reporting data about TPPs. 

• States are uniquely positioned to develop and deploy surveys of teachers and employers, and 
state-led surveys will deliver greater comparability of the resulting data. 

• ED should find ways to incentivize and support state reporting of teacher preparation quality 
that goes beyond simple compliance to meeting the needs of diverse stakeholders. 
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• ED should find ways to provide infrastructure and incentive for cross-state data sharing; 
currently states are not able to collect, report, or use complete information about teachers who 
leave or come into the state. 

• Regulations should support the development of data literacy skills among diverse stakeholders. 


Quality data about teacher preparation programs should be used within a culture of data use to inform 
continuous improvement and transparency, not only as a hammer meant to blame and punish. 


DQC Experience and Expertise on Teacher Data 

DQC believes that when stakeholders— including state leaders, educators, and parents— are empowered 
with education data, they make decisions that help improve system performance, increase 
transparency, and most importantly improve student achievement. To achieve this vision, DQC supports 
state policymakers and other key leaders to promote the effective use of data to improve student 
achievement. Launched in 2005 by 10 founding partners, DQC now works to realize the vision of an 
education system in which all stakeholders— from parents to policymakers— are empowered with high- 
quality data from the early childhood, K-12, postsecondary, and workforce systems to make decisions 
that ensure every student graduates high school prepared for success in college and the workplace. DQC 
is a nonprofit, nonpartisan, national advocacy organization based in Washington, DC. 

Since 2009, DQC has been surveying the states on the 10 Actions to Ensure Effective Data Use, 1 asking 
questions about what data they collect and how they are using them to improve student achievement. 
The DQC survey is a roadmap for states developed in collaboration with national partners and the states 
themselves. As such, the survey has tremendous buy-in and is used as a tool for charting progress in 
developing and using longitudinal data systems nationwide. It is sent annually to governors' offices and 
is completed through collaboration of staff from governors' offices, state education agencies, state 
workforce agencies, and higher education agencies. 

Included in our 10 State Actions are recommendations for sharing data annually about teacher 
performance with the institutions and programs that trained them, 2 and for creating quality data 
linkages between teachers and students. 3 Additionally, in partnership with the American Association of 
Colleges for Teacher Education, the Council of Chief State School Officers, and the National Council for 
Accreditation of Colleges for Teacher Education (now part of the Council for the Accreditation of 
Educator Preparation), DQC created a template to guide the process of creating data feedback loops for 
TPPs. 4 


Based on our annual survey, we have seen that sharing quality data with TPPs has been one of the most 


1 The Data Quality Campaign's 10 State Actions 

2 The Data Quality Campaign's State Action 9 ; 2014 Survey Instrument 

3 The Data Quality Campaign's roadmap for a high-quality teacher student data link 

4 DQC, AACTE, CCSSO, and NCATE data-sharing template 
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challenging of the 10 State Actions for states to meet in the past, but the number of states reporting 
that they share data with in-state teacher preparation programs jumped from 6 in 2011 to 22 in 2014. 
This demonstrates impressive leadership and will on the part of the states to provide high-quality 
feedback loops to their preparation programs. Part of the challenge of implementing these feedback 
loops, however, has been a lack of will, based largely on concerns that the data will be used as a 
hammer— to punish— rather than as flashlight— to inform continuous improvement and better 
decisionmaking among TPPs, state education agencies, district leaders, and aspiring teacher candidates. 
Further, while states have made tremendous strides building the data capacity needed, we hear 
anecdotally that some teacher preparation programs still do not receive the data that they want and 
need to improve from the state. For these reasons, it is critical that these regulations support states in 
ensuring that the data are trustworthy, transparent, and available to inform decisionmaking— not only 
accountability. 


Build Trust through Quality Data 

As in our 2011 comments, 5 DQC asserts that trust in the validity and reliability of the data used to 
measure TPP quality is critical, and therefore states must have in place a high-quality teacher-student 
data link (TSDL) to engender trust in the use of teacher data. While we do not take a position on 
whether or not the data should be used for TPP accountability as outlined in ED's draft regulations, if 
that it is the intent of data use it is even more important that the data be trusted by and useful to 
educators and the programs that train them. 


If data are to be used for high-stakes decisionmaking, the data must be trusted— accepted as valid, 
reliable, and fair. The linchpin of engendering trust in the data, and therefore in the system of 
measuring and reporting the quality of TPPs is that states must have a high-quality Teacher Student Data 
Link that reliably links teachers, students, and courses in ways that capture the complex connections 
that exist in the contemporary school setting. Therefore data systems and related policies must be able 
to: 

• Establish a statewide definition for teacher of record that captures the realities of the 
classroom. Students learn from numerous teachers and staff, in traditional and online formats. 
Efforts to understand educators' impact on students must reflect this reality and address 
classroom realities including team teaching, students who are pulled out for extra support, and 
virtual learning environments. These complicated relationships must be reflected in policies that 
allocate responsibilities for student learning by clearly defining teacher of record. Simple 
definitions, like "the teacher who assigns the grade" are not sufficient. 

• Link multiple teachers and students. Just as the teacher of record definition must reflect the 
realities of the contemporary classroom and the multiple educators that have an effect on 
student learning, so too must these relationships be accurately captured in the data system. 


5 Data Quality Campaign's Comments to Share Knowledge on States Data Capacity, 2011 
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Robust data linkages can link multiple teachers to each student per course, and are linked 
(updated) multiple times a year. 

• Capture schedule changes. Throughout the year, students' schedules often change as the result 
of the addition of a course section, dropping a class, or transferring to a new school. However, 
states typically only collect a snapshot of scheduling data at certain points in the year — and 
often just once at the start of the year. This practice fails to capture enough information to 
accurately record the time period for which a teacher is responsible for a student throughout 
the year. 

• Enable teachers to review their rosters for accuracy: To ensure data quality and assure teachers 
that decisions are being made based on accurate information, teachers must be given an 
opportunity to verify their rosters of students and submit corrections. This is a critical step in 
engendering educator trust in the validity and quality of data that links teachers to their 
students' outcomes. 


Without high-quality links between teachers and students, data used to demonstrate TPP quality based 
on graduate performance in the classroom will lack reliability, accuracy, and trust. 

DQC believes, based on surveys of the states since 2009, that many states currently implemented a high- 
quality TSDL, and that all states have implemented at least a basic TSDL needed to provide feedback to 
TPPs and data to the public. Nearly all states report having at least basic necessary linkages and 
infrastructure to produce indicators of student achievement growth attributed to educators. 

Current (2014) state capacity to match individual student data with individual teacher data 6 : 

• 44 states have implemented a teacher-student data link. 

• 34 states have in place a statewide definition of "teacher of record." 

• 33 states can connect more than one educator per student. 

• 31 states collect data more than once per year. 

• 26 states have a process in place for teachers to verify rosters. 

Building data infrastructure alone is not enough. To engender trust and support of the efficacy of the 
goals of ED's proposed regulations, it is critical that states continue their efforts to improve teacher- 
student data linkages, related policies, and use. In its final regulations, ED should emphasize the need 
for a high-quality teacher student data link comprised of the four critical components above as critical to 
ensuring that the data used to determine the quality of TPPs is effective, accurate, reliable, and trusted. 

Transparency about why data is collected and how it is governed is also critical to building trust and 
ensuring quality of these data. As with any data collection or use, it is imperative to have processes in 


6 DQC's 2014 Analysis of State Promising Practices in Defining Teacher of Record and Linking Teachers and Students 
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place that protect the privacy and security of this data. 

Inform Continuous Improvement 

DQC applauds ED's intention to ensure that TPPs have information about their graduates for the 
purposes of continuously improving educator training. States are increasingly looking to use their 
longitudinal data systems to inform strategies for improving educator effectiveness— and many are 
looking to improved teacher preparation quality as a means to improving their educator workforces. 

This is evidenced through DQC's annual survey, which demonstrates the growth from 6 states in 2011 to 
22 states in 2014 that are sharing multiple types of data, including graduate performance in the 
classroom based on student outcomes, with teacher preparation. A six-state effort led by the Council of 
Chief State School Officers called the Network for T ransforming Educator Preparation 7 is helping states 
focus on improving licensure and program approval policies, as well as using data to collect and report 
multiple measures of TPP quality for both continuous improvement and accountability purposes (DQC is 
an NTEP national partner). 

In addition to these efforts, DQCs annual survey demonstrates that as of 2014, a majority of states have 
the capacity to report multiple types of information about teachers to TPPs, and nearly half report data 
about how teachers perform in the classroom as measured by student achievement: 

• 39 states automatically share some types of data annually with in-state TPPs 


o 37 states share educator certification data (10 states share individual teachers' current 
level of certification; 9 states share aggregate numbers of teachers receiving initial 
certification disaggregated by preparation program; 18 states share both types of data) 
o 25 states share current school placement data, such as teaching in a high-poverty 
school (11 states share individual teachers' characteristics of current school placement; 

5 states share aggregate characteristics of current school placement; 9 states share both 
types of data) 

o 25 states share current employment status data, such as information on whether a 
teacher stayed at the same school, moved to a different school in the district, or taught 
in a different district in the state (8 states share individual teachers' current 
employment status in year two and year three following program completion; 6 states 
share aggregate current employment status for educators one, two, and five years after 
program completion; 11 states share both types of data) 
o 22 states share information about how teachers perform in the classroom as measured 
through their students' performance and course data— up from just six states in 2011 (5 
states share information about how individual teachers perform; 8 states share 
aggregate information about how teachers perform; 9 states share both types of data) 


7 Network for Transforming Educator Preparation 
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Further, recent evidence from Tennessee 8 demonstrates that the ability to share multiple types of data 
about graduates with TPPs provides those programs with valuable information for improving practice 
and producing higher quality teacher candidates. TPPs will best be able to make these types of targeted 
improvements in the ways they prepare teacher candidates when they have information about the 
performance of graduates by program, rather than by potentially diluted or averaged results by 
institution. In our 2010 data sharing template 9 created in partnership with AACTE, CCSSO, and NCATE, 
DQC highlighted the importance of sharing data at the program level. Therefore, DQC applauds ED's 
intention to report data at the program level rather than only at the institution level. 


The 2010 template referenced above is intended to be used by state leaders to facilitate conversations 
among multiple stakeholders, and especially between the leaders of K-12 agencies, postsecondary 
agencies, and heads of teacher preparation programs. DQC believes that it is critical to engage multiple 
stakeholders upfront to develop the key questions and indicators desired for creating transparency and 
providing quality data resources, as is the intent of ED's proposed regulations. Therefore, DQC applauds 
ED's intention to ensure that multiple stakeholders are engaged in the process of developing the system 
for sharing and reporting data about TPPs. 


Foster Comparability through State Leadership 

ED's proposed regulations intend to measure TPP quality through teacher and employee survey data, 
which provides a qualitative, measure of outcomes, therefore providing a more robust picture of 
candidate preparedness for the classroom. DQC proposed and provided a template for teacher 
satisfaction surveys in our 2010 partnership with AACTE, CCSSO, and NCATE 10 . To date, few, if any, 
states have yet to act on administering teacher quality/satisfaction surveys state-wide. DQC believes 
that states are uniquely positioned to develop and deploy surveys of teachers and employers, as states 
currently possess data about teacher licensure and employment placement. State-led surveys will 
ensure greater comparability of the resulting data, creating a clearer picture for both TPPs and the 
public of the range in the quality of TPP outcomes 

In addition, states should also take the lead in developing quality, accessible, and useful publicly 
available report cards of TPP quality. Reports of these kind must serve many stakeholders, including: 

• TPPs for cross-state comparison 

• State leaders for monitoring the teacher workforce 

• Principals for making hiring decisions 

• Aspiring teacher candidates for selecting a program 


8 Tennessee Improves Teacher Preparation Programs Through Report Cards 

9 DQC, AACTE, CCSSO. and NCATE data-sharing template 

10 DQC, AACTE. CCSSO, and NCATE data-sharing template 
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Therefore it is critical that states prioritize stakeholder engagement processes and develop report cards 
that are not only focused on compliance or even simply ranking programs, but instead answer multiple 
questions about teacher preparation quality that are useful to the above stakeholders and the public. 
Some state have begun to pursue high-quality public reporting for the information about the quality of 
K-12 11 schools, so there is an opportunity to learn from those efforts. Therefore, DQC encourages ED to 
find ways to incent and support state reporting of teacher preparation quality that goes beyond a simple 
compliance exercise to meeting the needs of diverse stakeholders. 


Opportunities for New Leadership 


There are additional supports and incentives states will need to create the highest quality plan for 
improving the quality of teacher preparation programs. These remaining challenges offer opportunities 
for new leadership among the states and from the ED. 

Cross-State Data Sharing 

One critical challenge to providing a robust, complete picture of TPP quality is the current inability of 
states to share information across state lines. For example, 21 percent of teachers who receive initial 
teaching licenses or certificates were trained in another state, yet states have no capacity 12 to link those 
teachers and their performance in the classroom back to the institutions that prepared them. Therefore, 
DQC encourages ED to find ways to provide infrastructure and incentive for cross-state data sharing as 
currently states are not able to collect, report, or use complete information about teachers who leave or 
come into the state. 


Capacity to Use Data 

Another critical consideration is the capacity of stakeholders at TPPs, as well as at state agencies and in 
districts, have the capacity to make use of information about teacher preparation quality. All state, 
national, and federal efforts should prioritize data literacy and capacity to use data at TPPs, state 
agencies, and among district and school leaders. Making sure that stakeholders are able to effectively 
use the information (data literacy 13 ) on program quality will realize the data's potential to act as a tool 
to improve teacher preparation, as well as hiring and placement decisions at the local level. Data literacy 
among diverse stakeholders who are interested in teacher preparation quality also engenders greater 
trust in the information, and helps ensure that the information are used not only for accountability, but 
for continuous improvement and improved decisionmaking. Therefore DQC believes that regulations 
should support the development of data literacy skills among diverse stakeholders. 

In Conclusion 


11 DQC's Brief on Quality Public Reporting 

12 DQC's Breaking Down Silos Report on Cross-State Data Sharing 

13 DQC Primer on Educator Data Literacy 
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States are uniquely positioned to provide feedback to teacher preparation programs on the 
effectiveness of the teachers they train, and to provide information to the public about the quality of 
TPPs. This work requires significant data capacity to reliably and securely link teachers with their 
students' achievement and growth data with the state's teacher preparation programs. While there will 
be challenges ahead, DQC asserts that most states have the capacity, and critically the will, to 
implement transparent, quality report cards and feedback loops that can improve the quality of teacher 
preparation programs. While cross-state data sharing remains a challenge, state efforts to provide 
robust, quality data about graduate performance in the classroom and other measures to TPPs is a 
critical step in continuously improving the educator workforce and ultimately, student achievement. 




